Visually-Represented Results To Search Queries In Rich Media Content

ABSTRACT

When executed, a computer program product generates a graphical user interface that renders results that are responsive to a search query of a rich media file. The graphical user interface includes a chronological representation of the rich media file, one or more occurrence markers along the chronological representation corresponding to actual occurrences of a desired term at an indicated chronological location in the rich media file, and an execution icon configured to launch a rich media application that renders a relevant portion that is responsive to the search query.

CROSS REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. Provisional Application No.60/740,276, filed Nov. 29, 2005 and titled “Visually-Represented Resultsto Search Queries in Rich Media Content,” which is incorporated byreference.

TECHNICAL FIELD

This document relates to accessing rich media content.

BACKGROUND

The growth of communications networks, such as the Internet, enablesaccess to wide varieties of content.

SUMMARY

In one general sense, content may be rendered by receiving a rich mediafile, analyzing the rich media file to identify a term within the richmedia file, associating the term with one or more time values specifyinga chronological location in the rich media file for the term, receivinga search query, relating the search query to the term, displaying achronological representation of search results in response to relatingthe terms to the search query, and enabling a user to interact with thesearch results to launch a rich media application that renders arelevant portion of the rich media file that is responsive to the searchquery.

Implementations may include one or more of the follow features. Forexample, displaying the chronological representation may includedisplaying one or more text transcripts associated with the searchresults or displaying a multi-occurrence icon representing two or moreoccurrences of the desired term that occur within a predeterminedchronological proximity. Displaying the chronological representation mayinclude indicating a number of occurrences of the desired term withinthe predetermined chronological proximity.

Receiving the rich media file may include receiving at least one ofaudio or video content. Enabling the user to interact with the searchresults may include enabling the user to interact with an execution iconconfigured to render the relevant portion by beginning to render therelevant portion at a predetermined amount of time in advance of theactual occurrence of the desired term such that rendering includescontextual content in addition to the desired term. Rendering therelevant portion with the contextual context may include selecting asufficient amount of the contextual content to make the rendering of thedesired term meaningful or selecting the contextual context based onbreaks in content or a rendering of sound effects that are associatedwith segments of a program to identify the contextual content.

Segment icons responsive to the search query may be rendered that arestructured and arranged to indicate different segments in the rich mediafile. The user may be enabled to render the rich media file at auser-specified chronological location relative to the actual occurrenceof the desired term.

Implementations of any of the techniques described may include a methodor process, an apparatus or system, or computer software on acomputer-accessible medium. The details of particular implementationsare set forth below. Other features will be apparent from thedescription and drawings, and from the claims.

DESCRIPTION OF DRAWINGS

FIG. 1 illustrates a graphical user interface (GUI) configured to rendersearch results of a rich media file.

FIG. 2 illustrates a GUI configured to render text transcriptsassociated with search results of a rich media file.

FIG. 3 illustrates a GUI configured to render multi-occurrence iconsrepresenting two or more occurrences that occur within a predeterminedchronological proximity in response to a search query.

FIG. 4 illustrates a GUI configured to render an image from a video filewhen results responsive to a search query are rendered.

FIG. 5 illustrates a GUI configured to render segment icons thatindicate different segments in the rich media file as results responsiveto a search query are being rendered.

FIG. 6 illustrates a GUI with a supplemental icon structured andarranged to render the rich media file at a user-specified chronologicallocation relative to the actual occurrence of the term.

FIG. 7 illustrates a communications system structured and arranged toenable a client to display a chronological representation of searchresults of rich media files in response to transmitting a search queryto a host.

FIG. 8 is a flow chart of an exemplary process by which a user isenabled to interact with search results to launch a rich mediaapplication that renders a relevant portion that is responsive to asearch query.

FIG. 9 is a flow chart of an exemplary process by which a user isenabled to interact with search results to launch a rich mediaapplication that renders a relevant portion that is responsive to asearch query for words within a movie database.

DETAILED DESCRIPTION

Rich media files (e.g., video or audio content) may be made moreaccessible by configuring content to be searchable such that a GUIenables a user to launch a rich media application that renders arelevant portion in response to a search query. While volumes of richmedia content are available, accessibility of the rich media content maybe improved by enabling users to access relevant portions of content(i.e., the portions of a rich media file that are responsive to thesearch query).

For example, a rich media file may be received by a host and analyzed toidentify terms within the rich media file. Each term may be associatedwith one or more time values specifying a chronological location in therich media file for the term. A search query is related to the term anda chronological representation of the search result is displayed inresponse to relating the term to the search query. A user is enabled tointeract with the search result to launch a rich media application thatrenders a relevant portion of the rich media file that is responsive tothe search query.

In one implementation, a GUI renders results in response to a searchquery of a rich media file. The GUI includes a chronologicalrepresentation of the rich media file, one or more occurrence markersalong the chronological representation corresponding to an actualoccurrence of a desired term at an indicated chronological location inthe rich media file, and an execution icon configured to launch a richmedia application that renders a relevant portion of the rich media filethat is responsive to the search query.

For example, a user researching news coverage of new medicines maysearch video coverage of recent developments. Using the GUI, the usermay perceive which video files (i.e., which rich media files) includerelevant content. More precisely, a user may perceive a chronologicalrepresentation of which portions of a video file include relevantcontent. By indicating which video files include results that areresponsive to a search query, and by indicating which specific portionsof the video file are relevant, the user may better understand how aparticular search is relevant to the user's interest. This understandingmay, in turn, enable the user to save time and/or enable the user tobetter tailor the user's efforts.

FIG. 1 illustrates a GUI 100 configured to render search results of arich media file. The chronological representation 120 of a rich mediafile 110 is rendered as a horizontal timeline along the bottom of thefigure. The leftmost point of the timeline is chronologically moreproximate to the beginning of the rich media file, and the rightmostpoint of the timeline is chronologically more proximate to the end ofthe rich media file.

The selection indicator icon 130 indicates a chronological point ofreference that is used when an execution icon 140 is selected. Theoccurrence icons 150 illustrate occurrences of terms that are responsiveto a search query along the chronological representation 120. Executionicon 140, shown as a play button with a triangle in between “fastforward” and “rewind” controls, enables a user to launch a rich mediaapplication that renders a relevant portion of a rich media file (e.g.,a portion at the indicated point along the chronological representation120).

A source information line 160 displays a source of the content, a filename, and/or other identifying features. A subject information line 170displays a brief description of the subject of the file. As shown, thesubject information line 170 includes a date for the result as “Sep. 15,2005” and proceeds with an exemplary summary “Lorum ipsum dolor sit amet. . . ” The subject information line 170 could have included anothersummary such as “Recovery efforts to Hurricane Katrina have been aidedby the Coast Guard.” A timer 180 displays a chronological length of thechronological representation 120. A search results line 190 displays anumber of overall search results to the search query and a numericalindication of the results shown on the page. A sorting line 195 displaysa basis by which results are displayed (e. g., date or relevance).

Generally, FIGS. 2-6 illustrate different implementations of a mediaplayer application (e.g., GUI 100 in FIG. 1) configured to displaychronological representations of search results. For example, FIG. 2illustrates a GUI 200 configured to render text transcripts 220associated with search results of a rich media file. The texttranscripts 220 include a brief text excerpt of the terms before andafter an occurrence of a term appearing in a search query.

FIG. 3 illustrates a GUI 300 configured to render multi-occurrence icons310 and 320 representing two or more term occurrences within apredetermined chronological proximity that are responsive to a searchquery. Each of the multi-occurrence icons 310 and 320 features anumerical indication 330 of the number of term occurrences associatedwith the icon.

FIG. 4 illustrates a GUI 400 that is configured to render an image froma video file when results that are responsive to a search query arerendered. The search results include still frames 410 that areassociated with occurrence icons 420.

In one implementation, a host receives a rich media file and analyzesthe imagery within the rich media file. The host may analyze the richmedia file for certain classes of images or desired subject matter(e.g., media personalities). For example, a host operated by a searchprovider may scan frames from a video file to identify overlaid graphicsafter determining that users often find overlaid graphics to featureparticularly meaningful content (e.g., polling results in political newsprograms). As a result, when the host provide results that include thevideo file with the specified classes of images and/or subject matter,the host may provide the results in such a manner as to enable displayof specified images. In one configuration, the host only providesspecified classes of images that lie within a specified proximity to aterm appearing in a search query. In another configuration, the hostprovides the specified classes of images irrespective of proximity to aterm appearing in a search query.

FIG. 5 illustrates a GUI 500 that is configured to render segment iconsthat are responsive to a search query as being arranged to indicatedifferent segments in the rich media file. The segment icons 510 may berendered to display breaks in content or sounds that are associated withconstituent parts of the rich media file.

In one implementation, the rich media file is received with an index ofchapters with or with metadata representing a location of content withinthe file. For example, a publisher may transmit a rich media file withsegment titles, indicated time(s) and/or file location information(e.g., a second chapter begins 10 kb into a 50 kb file).

FIG. 6 illustrates a GUI 600 with a supplemental icon structured andarranged to render the rich media file at a user-specified chronologicallocation relative to the actual occurrence of the term. The supplementalicon 610 is associated with a user-specified chronological locationrelative to the actual term occurrence (e. g., each occurrence icon maybe associated with a launching mechanism in the form of a supplementalicon 610).

In one implementation, a host identifies meaningful contextualinformation for occurrences of a term. For example, a user may notdesire to begin rendering a rich media file at the immediate occurrenceof the term. Rather, a user may prefer to render the occurrence of theterm with sufficient contextual information so as to make the renderingof the actual occurrence of the term more meaningful. The host may beconfigured to return results so that selection of a particular resultbegins a specified amount of time (e.g., 10 seconds) in advance of anoccurrence, or at the beginning of a sentence, paragraph, or segment inwhich the term appears. Alternatively, the host may return informationdescribing potential beginnings so that the user may select when thecontent will begin. For example, a client may render a supplemental iconconfigured to let the user select whether (1) a preceding 10 seconds ofcontextual information, (2) the beginning of a sentence of contextualinformation, (3) the beginning of a paragraph of contextual information,or (4) the beginning of a segment of contextual information should beused. Since a host may not have access to a script for the rich mediafile, a paragraph may be identified by identifying signals (e.g., “Wereport an interesting development in the case of . . . ”), byidentifying pauses, and/or by identifying a relationship between termsin chronological proximity.

FIG. 7 illustrates a communications system 700 structured and arrangedto enable a client to display a chronological representation of searchresults of rich media files in response to transmitting a search queryto a host. The communications system 700 includes a client 710, anetwork 720, and a host 730. The client 710 may include one or moredevices capable of accessing content on the host 730. Although thecommunications system 700 is shown as a network-based system, the systemmay access media files residing in a standalone device or in a differentconfiguration. For example, an information kiosk may play content in theform of music encoded in a media file format.

Each of the client 710 and the host 730 may be implemented by, forexample, a general-purpose computer capable of responding to andexecuting instructions in a defined manner (e.g., a personal computer(PC)), a special-purpose computer, a workstation, a notebook computer, aPDA (“Personal Digital Assistant”), a wireless phone, a server, adevice, a component, other equipment or some combination thereof capableof responding to and executing instructions. The client 710 may beconfigured to receive instructions from, for example, a softwareapplication, a program, a piece of code, a device, a computer, acomputer system, or a combination thereof, which independently orcollectively direct operations, as described herein. The instructionsmay be embodied permanently or temporarily in any type of machine,component, equipment, storage medium, or propagated signal that iscapable of being delivered to the client 710 or the host 730.

In one implementation, the client 710 includes one or more informationretrieval software applications (e.g., a browser, a mail application, aninstant messaging client, an Internet service provider client, a mediaplayer, or an AOL TV or other integrated client) capable of receivingone or more data units. The information retrieval applications may runon a general-purpose operating system and a hardware platform thatincludes a general-purpose processor and specialized hardware forgraphics, communications and/or other capabilities. In anotherimplementation, the client 730 may include a wireless telephone runninga micro-browser application on a reduced operating system with generalpurpose and specialized hardware capable of operating in mobileenvironments.

The client 710 may include one or more media applications. For example,the client 710 may include a software application that enables theclient 710 to receive and display an audio or video data stream. Themedia applications may include controls that enable a user to configurethe user's media environment. For example, if the media application isreceiving an Internet radio station, the media application may includecontrols that enable the user to select an Internet radio station, forexample, through the use of “preset” icons indicating the station genre(e.g., country) or a favorite station. In another example, the controlsmay enable the user to rewind or fast-forward a received media stream.For example, if a user does not care for a track on a particularstation, the user may interface with a “next track” control that willqueue up another track (e.g., another song).

The network 720 may include hardware and/or software capable of enablingdirect or indirect communications between the client 710 and the host730. As such, the network 720 may include a direct link between theclient and the host, or it may include one or more networks orsubnetworks between them (not shown). Each network or subnetwork mayinclude, for example, a wired or wireless data pathway capable ofcarrying and receiving data. Examples of the delivery network includethe Internet, the World Wide Web, a WAN (“Wide Area Network”), a LAN(“Local Area Network”), analog or digital wired and wireless telephonenetworks, radio, television, cable, satellite, and/or any other deliverymechanism for carrying data.

Generally, the host 730 includes one or more devices configured todistribute digital content. For instance, a host 730 typically includesa collection or library of content for distribution. Alternatively, orin addition, the host 730 may convert a media source (e.g., a video oraudio feed) into a first feed of data units for transmission across thenetwork 720. The host 730 also may include an input/output (I/O) device(e.g., video and audio input and conversion capability), and peripheralequipment such as a communications card or device (e.g., a modem or anetwork adapter) for exchanging data with the network 720.

The host 730 may include a general-purpose computer having a centralprocessor unit (CPU), and memory/storage devices that store data andvarious programs such as an operating system and one or more applicationprograms. Other examples of a host 730 include a workstation, a server,a special purpose device or component, a broadcast system, otherequipment, or some combination thereof capable of responding to andexecuting instructions in a defined manner.

The host 730 may include playlisting software configured to manage thedistribution of content. The playlisting software organizes or enablesaccess to content by a user community. For example, the host 730 may beoperated by an Internet radio station that is supporting a usercommunity by streaming an audio signal, and may arrange a sequence ofsongs accessed by the user community.

FIG. 8 is a flow chart of an exemplary process 800 by which a user isenabled to interact with search results to launch a rich mediaapplication that renders a relevant portion that is responsive to thesearch query. For example, the operations shown in process 800 may beused to render the GUIs shown with respect to FIGS. 1-6. Also, althoughthe operations are described with respect to a system 805, representinga general purpose computer, the operations shown in process 800 may beperformed on a client and/or a host (e.g., the client 710 and the host730 described in FIG. 7).

Initially, the system 805 receives a rich media file (810). Receivingthe rich media file may include accessing a database of searchablemedia. The database may be located within the system 805 or stored in aremote location. For example, a broadcaster may publish an audio filewith one or more news stories by transmitting the audio file to anintermediary such as a search provider.

The rich media file is analyzed to identify terms within the rich mediafile (820). For example, the system 805 may detect an occurrence of aterm rendered as a spoken word or a type of image or sound. The system805 then may separate terms occurring within the rich media file so thatsubsequent operations may relate search queries to a term (e.g., asystem 805 may produce an index with a list of terms occurring within afile for aiding search queries).

The terms are associated with time values specifying a chronologicallocation within the file (830). If, for each file, a list of occurringterms is generated, the list may include time values for each of theoccurring terms so as to enable results to be retrieved that areresponsive to a search query. The terms may be grouped by proximity suchthat users may search for a particular sequence of terms or a proximityof a first term with respect to a second term (e.g., within 30 secondsof one another).

A search query is received (840). For example, a user may provide searchterms in a web-based interface to a search engine.

The search query is related to the terms (850). For example, the searchquery may include one or more strings of characters to be searched foroccurrence within the rich media files. The strings may match specificterm occurrences or may use metadata related to terms in a search querysuch that a search engine associates a string with related terms.

A chronological representation of search results is displayed inresponse to relating the terms to the search query (860). Thechronological representation may include occurrence markers atchronological locations associated with the search results.

The user is enabled to interact with the displayed search results (870).The display with the chronological representation may include optionsand controls such as an indication of a chronological location of astarting point of a rendering relative to an actual occurrence of searchresults. For example, the search results may be displayed in a web pagewith embedded media player controls.

FIG. 9 is a flow chart of an exemplary process 900 by which a user isenabled to interact with search results to launch a rich mediaapplication that renders a relevant portion that is responsive to thesearch query for words within a movie database. In general, the systemsand operations described with respect to FIG. 9 relate to the systemsand operations described previously with respect to FIGS. 1-8. However,process 900 illustrates one implementation of a user interacting with asystem 905 to retrieve search results within a movie database.

Initially, a movie database is received (910). The movie databaseincludes one or more movie files. The system 905 analyzes movie filesusing a speech-to-text engine, and generates an index of terms thatoccur in a movie file (920). The speech-to-text engine also records anindication of the frames with which the term(s) are associated (930).Other references may be used, such as a file byte marker (e.g., 100 kbinto a 500 kb file).

With at least some of the content in movie database available foranalysis, a user requests a search for occurrences of the word “jedi”within the movies database using a web-based search engine (940). Thesystem 905 searches for occurrences of the term that are similar to theword “jedi” within the index of terms (950). Movies with a term relatedto the term “jedi” are displayed within the search results (960). Forexample, a media player application with a movie time line displaysoccurrences of the term “jedi”. The media application is configured torender the relevant portion of the movie that is responsive to thesearch query so as to be meaningful. For example, the media playerapplication may be played just before occurrences of the term (970).

In one implementation, the system 905 analyzes a rich media file forterms that are similar in sound. A search query for the word “jedi,” forexample, may result in search results that also relate to the phrase“jet fly” within the rich media, where “jet fly” is determined torepresent an alternative to an audio instance of “jedi.”

Other implementations are within the scope of the following claims. Forexample, a multi-occurrence icon may be configured to reflect the numberof occurrences or the likely degree of relevance. To this end, amulti-occurrence icon may be wider to reflect the duration over whichterms are located or more intense (e.g., taller) to reflect a largernumber of terms that appear.

Similarly, an icon (e.g., a multi-occurrence icon) may be modified toreflect the occurrence of metadata in the rich media file that isdetermined to be relevant to the search query. For example, if a termfound in a search query only appears once, but the relevant excerptincludes a large number of metadata terms related to the term found inthe search query, a narrow blue indicator may be used to reflect theoccurrence of the term found in the search query and a red band outsideof the narrow blue indicator may be used to reflect the occurrence ofmetadata terms. If text transcripts are supported, a firstdistinguishing characteristic (e.g., a blue font color) may be used torender the term found in the search query and a second distinguishingcharacteristic (e.g., a red font color) may be used to render themetadata terms appearing in the text transcript related to the relevantportion.

A host may scan a rich media file for distinguishing audiocharacteristics used to distinguish between segments. For example, ahost publishing audio files from NBC (the National Broadcasting Company)may use the presence of NBC's distinctive three tone clip to indicate abreak in content.

The described systems, methods, and techniques may be implemented indigital electronic circuitry, computer hardware, firmware, software, orin combinations of these elements. Apparatus embodying these techniquesmay include appropriate input and output devices, a computer processor,and a computer program product tangibly embodied in a machine-readablestorage device for execution by a programmable processor. A processembodying these techniques may be performed by a programmable processorexecuting a program of instructions to perform desired functions byoperating on input data and generating appropriate output. Thetechniques may be implemented in one or more computer programs that areexecutable on a programmable system including at least one programmableprocessor coupled to receive data and instructions from, and to transmitdata and instructions to, a data storage system, at least one inputdevice, and at least one output device. Each computer program may beimplemented in a high-level procedural or object-oriented programminglanguage, or in assembly or machine language if desired; and in anycase, the language may be a compiled or interpreted language. Suitableprocessors include, by way of example, both general and special purposemicroprocessors. Generally, a processor will receive instructions anddata from a read-only memory and/or a random access memory. Storagedevices suitable for tangibly embodying computer program instructionsand data include all forms of non-volatile memory, including by way ofexample semiconductor memory devices, such as Erasable ProgrammableRead-Only Memory (EPROM), Electrically Erasable Programmable Read-OnlyMemory (EEPROM), and flash memory devices; magnetic disks such asinternal hard disks and removable disks; magneto-optical disks; andCompact Disc Read-Only Memory (CD-ROM). Any of the foregoing may besupplemented by, or incorporated in, specially-designed ASICs(application-specific integrated circuits).

It will be understood that various modifications may be made withoutdeparting from the spirit and scope of the claims. For example,advantageous results still could be achieved if steps of the disclosedtechniques were performed in a different order and/or if components inthe disclosed systems were combined in a different manner and/orreplaced or supplemented by other components. As another example, ascreen name is used throughout to represent a unique identifier of anaccount, but any other unique identifier of an account may be used whenlinking accounts. Accordingly, other implementations are within thescope of the following claims.

1-21. (canceled)
 22. A system for processing search queries, the systemcomprising: at least one processor; and a storage medium storing aplurality of instructions, the plurality of instructions beingexecutable by the at least one processor for: receiving a search querycomprising a search term; determining an association between the searchterm and a media file; generating a visual representation of the mediafile with a plurality of indications corresponding to a plurality oftemporal locations in the media file where the search term occurs;receiving a selection of one of the plurality of indications; andrendering a portion of the media file in response to the selection. 23.The system of claim 22, wherein the portion of the media file comprisessegments of the media file before and after the temporal locationcorresponding to the selected indication.
 24. The system of claim 23,wherein the segments are selectable by a user.
 25. The system of claim23, wherein the segments comprise predetermined amounts of time.
 26. Thesystem of claim 23, wherein the segments are selected based on sectionmarkers in the media file.
 27. The system of claim 22, furthercomprising a plurality of instructions being executable by the at leastone processor for displaying text corresponding to the selectedindication, the text comprising the search term.
 28. The system of claim27, wherein the text is derived from a transcript associated with themedia file.
 29. The system of claim 28, wherein the text comprisesportions of the transcript before and after the temporal locationcorresponding to the selected indication.
 30. The system of claim 22,further comprising a plurality of instructions being executable by theat least one processor for generating a plurality of visualrepresentations of a plurality of media files, each visualrepresentation comprising an indication corresponding to a temporallocation in one of the plurality of media files where the search termoccurs.
 31. The system of claim 22, further comprising a plurality ofinstructions being executable by the at least one processor fordisplaying an icon that represents a number of temporal locations in themedia file where the search term occurs.
 32. A computer-implementedmethod for processing search queries, the method comprising thefollowing operations performed by at least one processor: processing asearch query comprising a search term; identifying a plurality of mediafiles based on the search term; generating an electronic visualrepresentation of a media file, from the plurality of media files, withan indication corresponding to a temporal location in the media filewhere the search term occurs; and rendering a portion of the media filein response to a selection of the indication.
 33. The method of claim32, wherein the portion of the media file comprises segments of themedia file before and after the temporal location corresponding to theselected indication.
 34. The method of claim 33, wherein the segmentsare selectable by a user.
 35. The method of claim 33, wherein thesegments comprise predetermined amounts of time.
 36. The method of claim33, wherein the segments are selected based on section markers in themedia file.
 37. The method of claim 32, wherein the operations performedby the at least one processor further comprise displaying textcorresponding to the selected indication, the text comprising the searchterm.
 38. The method of claim 37, wherein the text is derived from atranscript associated with the media file.
 39. The method of claim 38,wherein the text comprises portions of the transcript before and afterthe temporal location corresponding to the selected indication.
 40. Themethod of claim 32, wherein the operations performed by the at least oneprocessor further comprise generating a plurality of visualrepresentations of the plurality of media files, each visualrepresentation comprising an indication corresponding to a temporallocation in one of the plurality of media files where the search termoccurs.
 41. The method of claim 32, wherein the plurality of operationsperformed by the at least one processor further comprise displaying anicon that represents a number of temporal locations in the media filewhere the search term occurs.